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ABSTRACT 


Several Department of Defense agencies are currently investigating the use of 
distributed collaborative virtual environments (CVE) for the training of small dismounted 
infantry teams. If these systems are to be successful, they will have to do more than 
simply allow the team members to execute a task. In addition to assuring that essential 
training in the CVE transfers t> the real task, we will also have to ensure that aspects of 
team organization also transfer. In particular, we are investigating whether or not 
predicted emergent leadership, as measured by standardized personality tests, holds 
within a CVE or if aspects of the interface interfere. 

For a given “real-world” task domain a leader can be predicted based on 
personality traits of the individuals within the group. The interface utilized with a CVE 
may adversely affect these traits. In other words, predictive measures of leadership in the 
real world may not hold in a CVE. 

The study reported here will use this predictability to identify the expected 
emergent leader within a group and determine how the CVE interface affects the ability 
of the predicted individual to emerge as the leader. It is theorized that the lim itations of 
CVE interfaces (field of view, realism, etc.) will negatively impact the transfer of 
leadership personality traits into the virtual environment, but not to a degree that the 
limitation cannot be overcome. These lim itations may impact the group dynamics and 
the emergent leader may not necessarily be the predicted leader by personality traits. 
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I. INTRODUCTION 


A. MOTIVATION 

In today’s fast-paced world more people are dying on technology to bring them 
together, rather than actually gathering in a common location. Recent technologies, lik e 
high-speed Internet access virtual reality (VR), have opened new avenues for social 
interactions and collaboration efforts between remote locations. We see corporate 
executives using audio and video conferencing instead of travel. Network cameras are 
populating homes to keep families close. The military is using shared VR systems for 
training in lieu of costly real-world exercises. Collaborative Virtual Environments (CVE) 
are being used at research and design institutions. These technologies are changing the 
way people communicate, interact, and may be altering how we socialize. 

All of these technologies have led to significant research in how groups interact 
within a virtual environment (VE). Some of this research indicates that the interface, or 
immersion level, users have when working in a distributed VE (i.e. people are not co¬ 
located) can play a major role in how they interact. One example suggests that the user 
with the most immersive interface will emerge as the leader of the group when no leader 
is appointed. (Slater, Sadagic, Usoh, & Schroeder, 2000) This result may have been 
revealed because of the way VE interfaces affect a user’s level of immersion and 
presence within the VE. Presence and immersion are recognized as tightly coupled 
concepts that cannot always be well defined or measured. Slater, and others, suggest that 
presence is a psychophysical property that is user dependent and immersion is 
technological. (Slater & Wilbur, 1997) For example, the immersive qualities of a VE are 
tied to physical elements of the system like field of view; whereas, presence may be tied 
to how realistic the images appear to the user. Previous group interaction research 
conducted by Slater focused on the interaction of group members conducting a 
collaboration task using different interfaces and thus different immersion levels. This 
study also focused on distributed group collaboration when users have different 
immersion levels to readdress Slater’s research and to extend the dynamics of personality 
and group dynamics. 
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Hie dynamics involved during group collaboration work often have a significant 
effect on the outcome of the work produced. The movie, “12 Angry Men” (Lumet, 
1957), is a striking example of how a person’s personality, and how they communicate, 
can impact a group’s collaboration effort. The fil m does a good job of demonstrating the 
dynamics of group deliberations during a jury trial, the personality conflicts, the joint 
effort, and the functioning of several minds working together to find the truth. Taking a 
reverse perspective, it is easily seen how the dynamics of a group can impact any single 
participant. This social dynamic is often termed the “Abilene Paradox.” This condition 
exists when the members of a group succumb to a common goal that is contrary to each 
individual’s goal. As an example, imagine the result when one member of a group makes 
a suggestion they assume is the goal of the other group members, but different than their 
own desires. Now, the other group members assume the suggested goal meets the 
person’s desires and they each agree to the suggestion. They agree even though the goal 
is not their own personal desire. The result is a group that works to achieve a goal that no 
single member desires. 

There are many other factors involved in the dynamics of a group beyond those 
mentioned above. As an example, some research indicates that an attribute like gender 
can have an impact on group dynamics (Kimble & McNeese, 1987). It gets more 
complicated when variations on these factors are considered: group gender mix, leader 
gender, etc. Given the complexities of group dynamics and the amount of research that 
exists, shouldn’t equal consideration be given to the complexities of group dynamics in 
virtual environments? Specific attention must be paid to groups that collaborate within a 
shared virtual environment when the participants are not co-located. 

B. OBJECTIVE 

This study was designed to whittle away at some of the unknown qualities of 
collaboration within distributed virtual environments. To this end, the study looked 
specifically at the relationship between system interface, personality traits, and emergent 
leadership within a distributed collaborative virtual environment. 

It is widely known in social psychology circles that certain personality trait 
combinations are associated with emergent leadership in group interactions. These trait 
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combinations are highly dependent on the task domain. For instance, the traits required 
for emergent leadership within a social domain may not be the same as those within a 
particular work domain. Emergent leadership traits vary across different task domains, 
but are constant within that task domain. For a given domain, a leader can be predicted 
based on personality traits of the individuals within the group. To illustrate, if a group 
member exhibits traits associated with high verbalization and directive s kill s, they can be 
predicted with confidence to emerge as the leader of a group containing members with 
lower trait scores in those areas. This assumes the traits exhibited are associated with the 
task domain. (Schultz, 1990; Kimble & McNeese, 1987; Knowlton & McGee, 1994) 

This study used this predictability to identify the expected leader within a group 
and then test the effect of the CVE interface on the ability of the selected individual to 
emerge as the leader. It was theorized that the li mitations of CVE interfaces (field of 
view, degree of realism, etc.) would negatively impact the transfer of leadership 
personality traits into the virtual environment, but not to a degree that these lim itations 
could not be overcome. These lim itations may impact the group dynamics and the 
emergent leader may not necessarily be the predicted leader by personality trait. 

C. APPROACH 

To achieve this objective this study took the approach of designing a CVE that 
allowed a group to conduct a collaboration task that hey were familiar with. The task 
used was a navigation task within a natural tenain domain. Specifically, the participants 
were required to search, navigate, identify, and locate four targets within a virtual model 
of a natural outdoor scene. 

Hie virtual model replicated a section of land at the former Fort Ord military 
installation in Seaside, California. This model was used because it provided the ability to 
replicate as closely as possible a real world scene so that comparisons could be made 
with future work between group collaboration within a CVE and the “real-world” setting. 
Each participant completed a questionnaire to define two main characteristics associated 
with this study: personality and task expertise. A personality profile of the individual 
was recorded along with the individual’s self-rating of their expertise on the task. These 
two criteria were used to group the individuals for the study. Specifically, the groups 
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were arranged so that one participant could be predicted to emerge as the leader if the 
same task was conducted in the real world. 

During and after the experiment the participants were evaluated regarding the 
role they played within the group. Measures were taken to evaluate who in the group 
emerged as the leader. The emergent leader participant’s leadership and personality trait 
scores were analyzed for correlations with ratings of CVE interface devices to develop 
results. 
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D. BACKGROUND 


A. THE FIVE FACTOR MODEL OF PERSONALITY 

Several different personality measurement scales can be used to explain an 
individual’s personality. Some of the most popular are the Minnesota Multiphasic 
Personality Inventory (MMPI), the Myers-Briggs Type Indicator (MBIT), and the Five- 
Factor Model (FFM). Each tool has its own advantages/disadvantages and proponents. 
This study used a variation of the FFM called the NEO Five-Factor Inventory (NEO- 
FFI). The NEO-FFI is actually a reduced version of the revised NEO Personality 
Inventory (NEO PI-R), which has its roots in the FFM. 

The “FFM originated in initial works by Fiske (1949), Norman (1963), and 
Tuppes and Christal (1963), who produced a highly stable structure with five factors” 
(Salgado, 1997, p. 30). While most models are derived from theoretical perspectives, the 
FFM has a theoretically neutral position (Widiger, 1997). The NEO PI-R is a widely 
accepted measure of personality developed by Costa and McCrae, and assesses 
personality in terms of Neuroticism, Extraversion, Openness, Agreeableness, and 
Conscientiousness. The five personality factors are described in the following way: 

People who score high on NEUROTICISM typically report negative emotions 
such as worry, insecurity, self-consciousness, and tempermentalness (McCrae & 

Costa, 1987) whereas people with low Neuroticism are calm, self-confident, and 
cool (Salgado, 1997). 


EXTRA VERSION is the factor that describes people who are rated by their peers 
as “sociable, fun-loving, affectionate, friendly, and talkative” (McCrae & Costa, 
1987, p. 87) versus “reserved, timid, and quiet” (Salgado, 1997, p. 30). 

The final factor in this model is OPENNESS. Adjectives from lexical studies that 
describe this factor include “original, imaginative, broad interests, and daring” 
(McCrae & Costa, 1987, p. 87). “Openness defines individuals who are creative, 
curious, and cultured versus practical with narrow interests.” (Salgado, 1997, p. 
30) 
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People high in AGREEABLENESS are forgiving, lenient, sympathetic, agreeable, 
and softhearted, according to peer ratings (McCrae & Costa, 1987). Peers 
describe those low in Agreeableness in more negative terms: ruthless, 
uncooperative, suspicious, and stingy. 

Peers describe people high in CONSCIENTIOUSNESS as careful, well 
organized, punctual, ambitious, and persevering (McCrae & Costa, 1987). 
Conscientiousness includes both proactive (hardworking, ambitious) and 
inhibitive (dutiful, scmpulous) aspects (McCrae & Costa, 1989). 

These personality measurement tools are important because they can be used to 
closely predict which individual should emerge as the leader for a given group 
collaboration task. 

B. PERSONALITY MEASURES 

Some personality measurement techniques are more objective while others are 
subjective and prone to bias. A goal for this study was to use a technique that is 
standardized, and that also has high reliability, and validity. These attributes tend to 
define the best assessment techniques. Standardization, reliability, and validity ensure 
consistency for administration, consistency of results, and accuracy of measurement. The 
NEO-FFI has these qualities. As an example, Salgado (1997) while evaluating the 
validity and consistency of various personality measures for predicting job performance 
in the European community co nfir med that tests based on the Five-factor Model provide 
consistent results across both North American and European communities. The NEO PI- 
R was among the tests used and the NEO-FFI is fully based on the NEO PI-R. McCrae 
and Costa (1987) also compared the NEO PI-R measures between self-reports and peer 
ratings and found statistically significant validation of the measures. 

A primary method for assessment is the self-report inventory method. The self- 
report method asks people to report on themselves by answering questions about their 
feelings and behavior in a variety of situations. The person taking the survey must 
indicate how closely each item describes their own traits or how much they agree with 
each item. The standardized administration, scoring, and evaluation of the NEO-FFI 
allows for effective self-reporting. 
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As indicated above, the MBTI and MMPI are alternatives for personality 
assessment. The MBTI requires several hours to administer and evaluate and requires a 
trained psychological professional to interpret the scores. The MMPI determines 
personality traits of hypochondriasis, depression, hysteria, psychopathic deviate, 
masculinity- feminity, paranoia, psychasthenia, schizophrenia, hypomania, and social 
introversion. The MMPI is primarily used by clinical psychologists as a diagnostic tool 
for assessing personality disorders, but is also utilized as a vocational tool. Like the 
MBTI, the MMPI is extremely long to administer and require special training to interpret 
the results. 

The NEO-FFI, on the other hand, is easy to administer and interpret. The entire 
process can take less than 60 minutes per individual. It has been validated and used in 
previous research at the Naval Postgraduate School. For these reasons, the NEO-FFI fits 
the requirements of this study well. 
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III. APPARATUS 


A. NEO-FFI 

This study required an assessment tool that has high reliability and validity 
ratings, while also being easy to administer, easy to complete with minimal time 
requirements, and easy to interpret. The NEO PI-R, the predominant measure of the five- 
factor model of personality, was chosen for inventorying an individual’s personality since 
it meets all the above criteria. (Widiger, 1997) The NEO PI-R consists of 240 statements 
to which a person indicates their degree of agreement on a 5-point scale. The NEO PI-R 
is often referred to as a lexical five-factor model since it attempts to define personality in 
natural language terms. 

Buziak (2000, p. 24) states that “substantial research exists regarding NEO PI-R 
reliability and validity. Most importantly, the NEO PI-R has demonstrated consistent 
convergent and discriminant validity, as well as indicating how alternate models can be 
understood from the perspective of the five factor model.” He goes cn to note that many 
studies on personality measurement, including the MBTI, have used the NEO PI-R as a 
comparison to find overlaps because of the consistency in results. The use of the NEO 
PI-R for comparison is due to the high correlation with the five factor model, which does 
not rely on a particular theory of personality. (McCrae & Costa, 1987) 

The NEO Five Factor Inventory (NEO-FFI) is a 60-question subset of the full 
240-question NEO PI-R. The NEO PI-R’s additional length allows for more precise 
measurement and better false answer detection while the shorter length of the NEO-FFI 
required less time to administer. The authors of the NEO PI-R do not envision any 
significant changes in the near future; thus, it is a logical conclusion that there will be no 
major revisions planned for the NEO-FFI and it can be used with confidence. (McCrae & 
Costa, 1992) 

B. TASK EXPERTISE 

Establishing a profile of the individual personalities alone was not sufficient for 
this study of emergent leadership. As indicated above, specific personality traits are 
often effective within certain task domains. Since this study used a land navigation 
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exercise for the collaborative task, each individual’s expertise and experience within this 
task domain was profiled. As part of developing these profiles, a land navigation 
background questionnaire was developed and ut iliz ed. 

The questionnaire defined broad categories of expertise levels within the land 
navigation domain. By using a profile of each individual, the groups could be formed so 
that no single individual had a significant advantage on the task based solely on their land 
navigation expertise. This was important in order to ensure personality traits were the 
underlying force behind any emergent leadership. Expertise of a group leader plays an 
important role in land navigation tasks because the leader is looked upon for guidance 
and training (Stine, 2000). 

C. VIRTUAL ENVIRONMENT MODEL 

A distributed collaborative virtual environment was developed to support the 
collaborative task for this study. The distributed nature of the CVE presented a 
significant human factors issue in the experiments conducted by Slater (2000). To 
support this endeavor, the virtual model was represented on three high-end graphics 
computers over a Local Area Network (LAN). Each computer interface provided a 
different degeree of immersion similar to the Slater experiments. The individual 
immersive interfaces utilized a CAVE (CAVE Automatic Virtual Environment - 
originally developed at University of Illinois, Chicago) interface, a system with three 
large-screen monitors, and a standard computer monitor. All other interface tools, such 
as input devices, were kept identical. 

1. Virtual Model Description 

The shared virtual environment was created using Multigen-Paradigm Creator. 
The entire model was built to scale with overall dimensions of approximately 1000 by 
500 meters. Terrain elevation was developed in accordance with local topographic maps 
having contours every ten feet. A satellite image of the area was overlaid on the terrain 
map and helped to establish paths within the environment. Photographs of objects and 
vegetation were taken of the represented area and converted to objects and placed in the 
virtual model. Figure 1 shows a sample view of the virtual model, as the user would have 
experienced it. The end result of the model was an environment where the user, while 
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moving, experienced parallax and obscured visibility due to vegetation. Additionally, 
they could identify key landscape features, terrain elevation changes, and paths. 



Figure 1. Virtual Model as Seen by User. 

2. High-Immersion Station 

Hie workstation used by the participants that provided the highest level of 
immersion was implemented using a CAVE-like interface. It was composed of three (3) 
large rear projection displays each having an 81-inch diagonal measurement as depicted 
in Figure 2. The displays were connected together at a near seamless 45 degrees and 
almost completely fille d the user’s normal field of view. The virtual model was rendered 
in real-time via Multigen-Paradigm’s Vega software. The view point for each display 
was rendered using three 45-degree frustrums set at 45-degree offsets, which together 
presented a 135-degree graphical field of view. 
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Figure 2. High-Immersive Display Layout. 


Each display projector was driven by a dedicated high-end personal computer 
(PC). The computers uti liz ed the Vega software feature that allows one “master” 
computer to distribute scene rendering to multiple “slave” computers. All three 
computers were linked via a high-speed gigabit network and the master controlled the 
display synchronization. Table 1 shows the hardware configuration for each computer. 


Operating System 

Windows 2000 (SP2) 

Computer Processor 

Dual Intel Pentium HI, lGhz 

Memory 

2 Gigabytes 

Graphics Adapter 

Nvidia GeForce 3 

Frames per Second 

Greater than 25 


Table 1. High-Immersion Workstation PC Configuration. 
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3. 


Mid-Immersion Station 


The mid-level immersive station was rendered using the same software 
configuration as the high-immersion station; however, at this station the computer and 
display device were significantly different. This station utilized a single PC with multiple 
graphics adapters to render the scene to a single Panoram PV230 DSK display that 
integrates three (3) 15-inch high-definition LCD displays. The PV230 display provided 
the user with a high quality semi-immersive display of the virtual environment. Table 2 
shows the hardware configuration for the PC rendering to the PV230 display. 

4. Low-Immersion Station 

The low-level immersive station was rendered using an Intergraph computer with 
multiple graphics adapters, but only one display device actually rendered the virtual 
model. Rather than display three separate frustrums, this station only rendered a single 
45-degree frustrum sent to a single 21-inch display monitor. This configuration provided 
the user with a significantly reduced graphical and physical field of view. Table 2 shows 
the hardware configuration for the PC. 


Station 

Mid-Immersion Station 

Low-Immersion Station 

Operating System 

Windows 2000 (SP2) 

Windows NT 

Computer Processor 

Dual Intel Pentium II, 800 Mhz 

Dual Intel Pentium II, 450 Mhz 

Memory 

512 Megabytes 

512 Megabytes 

Graphics Adapter 

Three 3Dlabs Oxygen GVX1 

Wildcat 2000 

Frames per Second 

Approximately 10 

Approximately 20 


Table 2. Mid- and Low-Immersion Workstation PC Configurations. 


All other features of the user interface at each station were kept the same. Each 
participant sat at the workstation and used a keyboard and mouse to manipulate the 
virtual scene. Each model was rendered on the display at a high resolution of 1024 by 
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768 pixels. Additional information on hardware and software configuration is located in 
Appendix A. 

D. USER REPRESENTATION AND CONTROLS 

As is normal for distributed collaborative environments, the participant human- 
human interaction methods are severely lim ited compared to real world interactions. 
Slater’s experiment essentially lim ited the human interaction to voice communications. 
In comparison, this study used voice communication augmented with a “life- lik e” avatar 
with a lim ited set of controls including a form of pointing. These lim itations impact 
group collaboration effectiveness because people usually use other methods of 
communicating ideas and desires, such as touch, facial gestures, and body language. 

1. Voice Communication 

Voice communication between the participants was handled by “voice-over-IP”. 
Each user was provided an audio headset with an attached boom microphone. 
Additionally, each headset had controls to adjust audio level and microphone mute. A 
freely available software product, “Team Sound” by RedWired Software, connected each 
user to a dedicated server. To reduce network traffic and keep communication as natural 
as possible, all transmissions were controlled by a voice-activated feature of the software. 
For clarity, a minimal compression algorithm was used. Slight transmission delays were 
noted by users, although all easily adapted. 

2. Avatar Control and Movement 

Each user interacted with the virtual model via an avatar generated by Boston 
Dynamics DI-Guy software. This software provided a realistic representation of each 
user as a “soldier” carrying a weapon. The avatar motions and actions were controlled by 
keyboard commands. Each user could command their avatar to “turn left/right,” “stand,” 
“walk,” “walk backward,” “run,” “kneel,” “lay down,” and “aim rifle.” Figure 3 shows 
these various states. By commanding the avatar to “turn left/right,” “stand,” “walk,” 
“walk backward,” or “run” the user was able to navigate through the virtual environment. 
The “kneel” and “lay down” actions were available to allow the user to change the height 
of their viewpoint. Changing view height was helpful to the user in areas where they 
wanted to see under the low branches of trees. 
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Since this experiment required the participants to navigate together, the capability 
of each user to be able to point at objects was deemed necessary. The “aim rifle” feature 
met this requirement by allowing the user to point their rifle in a given direction. It 
should be noted that this is a common way to point under a dismounted infantry military 
exercise context. 



Figure 3. Avatar States. 


Hie only other control the participants had available was mouse control of 
viewpoint. The intent was to allow the user to be able to look around without changing 
their direction of movement. This emulates the notion of walking and turning your head 
to see something to your side. The mouse motions were translated into viewpoint 
changes in azimuth and elevation. 
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There were two negative artifacts of the avatar that were introduced as a result of 
software lim itations. The first being that the transitions between avatar states were not 
smooth. For instance, when a user commanded the avatar to “aim rifle” the action 
occurred immediately rather than affording a smooth transition of raising the rifle. Hie 
second artifact was that each user could not see their own avatar, with the exception of 
their rifle. Although seeing their own rifle was helpful, especially when trying to realign 
view direction with movement direction, it provided very little proprioceptive feedback. 

All of the character states of each user were transmitted to the other participants 
and the representative avatar would change state. Overall this provided a more realistic 
expression of each user than Slater’s study, although still a ‘Tar-cry” from what people 
experience in real-world collaboration. 
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IV. METHODOLOGY 


A. EVALUATING PARTICIPANT PERSONALITY 

As mentioned above, this research required profiling each participant’s 
personality. These profiles were used to form collaboration groups for the experiment 
task. Specifically, each group required one, and only one, individual with personality 
traits that would predict that this person would emerge as the group leader. The NEO- 
FFI provided the framework for profiling the potential participant personalities. In 
addition to forming groups based on personality, each participant’s expertise on the 
experiment task was also profiled. 

1. Survey Implementation 

The questionnaires for profiling personality and expertise were combined into one 
document. This helped to limit the number of separate documents the participants were 
required to complete. Although the profiling was conducted using one document, the 
questions were divided into separate sections so that the participant noticed the shift in 
question domains. 

NEO-FFI personality surveys from Psychological Assessment Resources, Inc. 
were used for documenting each individual’s personality traits. Because this study 
required participants that possessed specific personality traits, a large pool of perspective 
participants was needed. To meet this objective, the NEO-FFI survey was additionally 
converted to hypertext documents and made available via a secure web site. Care was 
taken in developing the web-based documents to ensure respondents understood how to 
complete the survey and their rights to privacy. For those individuals who did not desire 
to complete the personality survey via the web, information was provided on how to 
complete the questionnaires using the normal paper documents. 

2. Survey Documents 

Each of the survey implementations presented the instructions and questions in 
exactly the same wording and format. Figures 4 and 5 provide examples of each of the 
survey implementations. The only notable difference between the implementations was 
in the recording format. The hypertext document provided “radio buttons” adjacent to 
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the questions and responses were automatically recorded to an electronic database. The 
traditional paper documents had a page for questions and a separate page for recording 
the answers to the questions. The electronic database automatically keyed each user with 
a unique identification number. This number was used throughout the experiment to 
anonymously record information about each participant. Responses recorded via 
traditional paper documents were manually entered, by the principle investigator, into the 
electronic database, and a unique identification number was assigned. 


NEO Five-Factor Inventory (Form S) 

SD 

D 

N 

A 

SA 



i. r 

r 

r 

r 

r 

i 

am not a worrier. 

2. r 

r 

r 

r 

r 

i 

like to have a lot of people around me. 

3. r 

r 

r 

r 

r 

i 

don’t like to waste my time daydreaming. 

4. r 

r 

r 

r 

r 

i 

try to be courteous to everyone I meet. 

5. r 

r 

r 

r 

r 

i 

keep my belongings clean and neat. 


Figure 4. Question Format from Web-based Survey. 


1. I am not a worrier. 

2. I like to have a lot of people around me. 

3. I don’t like to waste my time daydreaming. 

4. I try to be courteous to everyone I meet. 

5. I keep my belongings clean and neat. 


Enter your responses here—remember to enter responses across the rows. 
SD=Str<mg1y Disagree; D = Disagree; N = Neutral; A = Agree; SA=Strongly Agree 


1 ©®®®® 
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3 ©®®®@ 
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Figure 5. Question Format from Paper-based Survey. After Ref. (McCrae & Costa, 1992). 
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All the questions on the survey required the respondent to make a decision on 
whether they agreed with a particular statement on a scale from “Strongly Disagree” to 
“Strongly Agree”. The NEO-FFI converts these answers to a numeric value used for 
scoring the various personality traits. It was these numeric values that were recorded in 
the database for later analysis. 

3. Survey Responses 

After completing the survey, each participant was advised they would be 
contacted to coordinate any further participation. This allowed time for the coordinator 
to review the surveys and establish the groups for the experiment. All communication 
after completing the survey was handled via email. 

A total of 67 people initiated the survey, but only 56 completed all required 
information and questions. The 56 respondents were all staff or students at the Naval 
Postgraduate School in Monterey, California. All respondents were over the age of 21; 4 
of the 56 respondents were female. 

4. Emergent Leadership Personality Traits 

The traits associated with emergent leadership for this study were related to the 
required navigation and communication tasks. The experiment focused primarily on 
communication over navigation. Previous research conducted for the Office of Naval 
Research (ONR), indicated that certain personality traits are more useful for certain tasks. 
That research divided tasks into broad categories: realistic, investigative, artistic, social, 
enterprising, and conventional. For this study, the required navigational and 
communicational tasks fit into the realistic and social domains - primarily the social 
domain. Based on the research for ONR, group performance for the assigned task should 
improve when the members of the group have traits that are high in conscientiousness 
and agreeableness. (Hogan, Raza, Sampson, Miller, & Salas, 1989) 

A number of studies have compared leadership and personality. In 1994, Hogan, 
Curphy, and Hogan, conducted an in-depth analysis of these studies and concluded 
emergent leadership is a specialized niche within the leadership domain that develops in 
small groups when no appointed leader exists. The result of the analysis Hogan, et al. 
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(1994) conducted on emergent leadership shows a correlation between emergent 
leadership and personality traits. Specifically, that the personality traits of high surgeney 
(sociability, social presence, etc.), conscientiousness, and emotional stability are 
correlated to emergent leaders. These traits closely map to high scores in extraversion 
and conscientiousness, and a low score in neuroticism on the NEO-FFI personality 
assessment, respectively. By combining these two results, this study anticipated that 
extraverted, conscientious, and emotionally stable people would emerge as leaders in a 
group that was made up of members that did not possess these traits. 

The NEO-FFI divides the scale for each of the five traits into categories: Very 
Fow, Fow, Average, High, and Very High. This study considered the terms “low” and 
“high” in previous research to equate to Fow/Very Fow, and High/Very High, 
respectively. Of the pool of 56 respondents, only 7 met the personality characteristic of 
“emergent leader”; specifically, high or very high scores in extraversion and 
conscientiousness, and a low or very low score in neuroticism. To provide a larger pool 
of potential leaders, another pass was taken through the database to find all the people 
that did not meet the personality characteristics, but differed only by an “average” score 
in one of the evaluated traits. For example, someone with an “average” score on 
extraversion, high score on conscientiousness, and low score on neuroticism would not 
have been among the original 7, but would have been included in the second pass. This 
second pass increased the “potential leader” pool by 12 people. 

5. Other Group Member Personalities 

Once the list of potential leaders had been generated, another pass was taken 
through the list of respondents to find prospective group members. On this pass the 
primary consideration was to identify those individuals that had personality traits 
opposite of those selected for the “emergent leader” pool. Specifically, the goal was to 
select people that had personality traits with the following characteristics: low 
extraversion, low conscientiousness, and high neuroticism. This pass provided only a 
handful of potential group members, which was not sufficient for this study. 

To broaden the list of potential group members who had personality traits 
opposite of the traits possessed by the predicted group leaders, a list was generated that 
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contained only people with average or lower extraversion scores. Extraversion is the trait 
with the highest correlation to emergent leadership (Hogan, Curphy, & Hogan, 1994). 
This provided a pool of 22 people that could be used as group members to be paired with 
the predicted leaders. Given the two lists, predicted leaders and followers, groups were 
formed based on the available schedules of the potential participants. A “best effort” was 
made to keep the group members task expertise at a common level so that any emergent 
leader would be based solely on personality attributes. Additionally, the groups were 
gender homogenous to alleviate any gender biasing. 

B. COLLABORATION TASK 

Each group was assigned a task that was designed to maximize collaboration 
among the group members. A land-navigation task was devised to achieve this objective. 
Land navigation was selected because the Naval Postgraduate School has conducted a 
series of studies involving cognitive modeling of group behaviors within real and virtual 
navigation tasks. Previous research had also indicated that while conducting small group 
navigation exercises, the members of the group were very involved in collaborating when 
they were disoriented. (Boswell, 2000) This provided a nice venue for developing a 
navigation task that purposely placed the participants in a disoriented state. In this way, 
extensive collaboration could be assured. 

1. Task Objective 

The general task objective was to search for targets or markers within the virtual 
environment, and when located, plot those locations on a map. During the experiment in¬ 
brief, each participant was provided guidance on this objective. An explanation of the 
collaborative and navigational tools was presented and the group as a whole had an 
opportunity to experience how to navigate and communicate within a practice 
environment. Prior to entering into the experimental environment, each participant was 
again advised that the goal of the group was to work together to identify the location and 
orientation of the group as a whole and to develop and implement a strategy for locating 
targets. Each participant was provided an outline map of the environment for marking 
purposes. It was left up to the group members to decide what roles, if any, each member 
would take to solve the problem. 
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2. Navigation Tools 

In all land-navigation tasks specific tools are required to achieve the objective. 
These tools provide the user the ability to identify current location, orientation to some 
known reference point, and relative scale of the environment. For this study, several 
tools were introduced to the users to assist in achieving their objective. As a whole, the 
group had the following set of tools available: 

• Compass 

• Overhead color imagery map depicting vegetation and roads 

• Terrain contour map with a compass rose identifying North 

• Distance traveled in meters indicator 

• Photos of key landmarks within the virtual environment 

• Map identifying locations of the key landmarks 

If any single person had all these tools available they would be able to move through the 
environment with confidence and be able to effectively identify their location. To ensure 
that each member of the group was fully involved in collaborating about the navigation 
task, each member was supplied with only a portion of the tools required to effectively 
complete the task. In this manner, each member was required to communicate what tools 
they had available for the task and how they could help solve the problem. The grouping 
and distribution of the navigation tools is described below in the variable manipulation 
section. 

3. Group Coordination 

Given the lim ited set of tools available to each group member, some level of 
coordination was required. During the in-brief for the experiment, each participant was 
provided guidance on what tools were available to the group as a whole, but not who 
possessed each of the tools. Additionally, the group was instructed that any movements 
and decisions would need to be made as a group. That is to say, that no one participant 
was free to roam around the environment alone. These attributes of the task induced 
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more collaboration among the participants on how to complete the task. It also increased 
the lik elihood of one participant emerging as a leader as final decisions were established. 

4. Group Member Anonymity 

The study required that participants in any one group not be unduly familiar with 
each other. Lester (2001) suggests this requirement must be adhered to keep the level of 
trust between participants to a level of “cognitive-based trust”. Without this lim itation 
the participants may have had different interactions due to anticipated actions of others 
caused by familiarity. Every group was developed through coordination by the principle 
investigator of each potential participant’s schedule. At no time during the recruiting 
process were any of the participants exposed to whom else may be participating in the 
experiment. Each group member was briefed individually prior to starting the 

experiment task and only told that there were two other members of their group. 

Anonymity was also provided during the group collaboration by assigning generic 
names to each of the group members. As all the participants had military experience and 
they were represented within the CVE as soldiers, traditional military communication 
callsigns were used for identification purposes. Each member was assigned the callsign 
of “Echo”, “Kilo”, or “Tango” throughout the experiment. Each participant was 
instructed to only use these callsigns for communicating. The callsign was also attached 
to corresponding avatar for identification purposes. 

C. VARIABLE MANIPULATION 

The primary purpose of this study was to ascertain the potential effects CVE 
interfaces can have on emergent leadership. When studying human factors related issues 
there can be some variables that are difficult if not impossible to control. Some of these 
variables may not even be recognized as a factor. The formation of the groups as 
described above was a direct attempt to min im ize any confounding factors. This should 
allow for the only independent variables to be the interface used by participants and the 
tools used for collaboration. 
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1. Rotating Interface 

The interface was the variable that was expected to have the highest potential 
impact on emergent leadership within the group. Each participant was exposed to each of 
the three different levels of immersion into the CVE. If the interface did have an affect 
on leadership, as suggested by Slater (2000), then the most immersive interface should 
produce more emergent leaders regardless of whether those people had traits that predict 
leadership. The other argument was to look at how often a predicted leader emerged as 
the leader regardless of the interface. By rotating each participant through all of the 
interfaces, both of these arguments could be studied. Thus, each participant was exposed 
to each of the three interfaces, or immersion levels, and evaluated for their role, 
specifically leadership role, while experiencing each interface. Additionally, during the 
rotation the anonymity of each group member was maintained. 

2. Rotating Tool Sets 

The tools for navigating were divided into three sets where any one set did not 
provide enough input to the user to accomplish the task on their own. In this way, each 
member was required to request assistance from the other members. The three sets of 
tools were divided in this way: 

• Compass, movement counter, and vegetation map 

• Compass rose, contour map, and photos of targets 

• Key feature location map and photos of key navigation objects 

Consideration was given to whether the tool set selection criteria was a dominant factor 
in who would emerge as the leader. To ensure that one set of tools was not more 
advantageous to the task and thus set one participant apart by providing a performance 
advantage, the tools were rotated between the different CVE interfaces. This rotation 
occurred between groups not between individuals. For example, one group may have 
tools sets 1, 2, and 3 combined with interfaces 1, 2, and 3 respectively. Each participant 
in this group was then rotated between each of the combinations. The next group would 
have tool sets 1, 2, and 3 combined with interfaces 2, 3, and 1 respectively. In this way, 
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any correlations that exist between interface, tool set, or personality and emergent 
leadership could be identified. 

D. MEASURING EMERGENT LEADERSHIP 

There is not always a well-defined method for determining who the leader is 
within a group. Previous research indicates that verbal communication can be used as an 
indicator of leadership (Kimble & McNeese, 1987; Slater, et al., 2000). These studies 
used different methods to quantitatively measure verbalization, but each looked at the 
amount of talking done by any individual and the nature of the talking. 

This study used a similar method to identify who within the group was perceived 
as the leader. Participants were required to complete a survey after conducting the 
navigation task using each of the different interfaces. The survey, as represented in 
Appendix B, consisted of a series of questions that required them to rate each person 
within the group, including themselves, on various attributes of the collaboration effort. 
Specifically, they rated each participant on talkativeness, quantity of feedback, quality of 
feedback, effectiveness of input, and overall involvement. For each rating, the rater was 
instructed to place a mark on a linear scale from “Low” to “High.” These group-peer 
evaluations were compiled as a subjective measure of who had emerged as the leader 
within the group for that portion of the experiment. After completing the interim survey, 
each participant was rotated to the next station to continue the experiment with one of the 
other CVE interfaces. 

After rotating through each immersion level and completing a survey for each 
trial, the participants completed a second survey (Appendix B) that queried the user on 
aspects of the overall experiment. This survey included as the last question a rating of 
each group member on overall leadership. All of these ratings were quantified and used 
during the analysis to determine who had emerged as the leader during each exercise. 
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V. ANALYSIS AND RESULTS 


A. EVALUATING EMERGENT LEADERSHIP 

Six groups of three people completed the initial personality survey and the CVE 
task. As indicated earlier, each of the participants evaluated the other people in their 
group, including themselves, on behaviors correlated with group leadership. These 
evaluations were subjective in nature and attempts were made to develop objective 
measures of emergent leadership. 

1. Recording Group Participation 

To help evaluate the group members for emergent leadership, the actions and 
behaviors of each participant were recorded on video and audio equipment at the various 
workstations. It was anticipated that these recordings could be reviewed after each 
session to produce objective measures for emergent leadership. Specifically, the intent of 
the recordings was to evaluate each individual for the following objective criteria. 

• Amount of time individual spent talking 

• Number of orders or suggestions given 

• Number of regroup or cooperation statements 

• Number of positive group statements given 

• Amount of time individual led the group formation within the CVE 

• Number of final decision made by individual 

These criteria could then be talked for each individual and entered into a formula for 
evaluating the individual as the emergent leader. Unfortunately, while reviewing these 
recordings, it was determined that some of the data was garbled or it was difficult to 
accurately identify which actions were taken by which participants. It was deemed 
inappropriate to use these recordings for creating objective measures due to the inability 
to determine which participant was speaking each utterance. However, the recordings 
were useful for improving the context of comments made by the participants on their 
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post-experiment questionnaires and interviews. This information can be applied to future 
research in small group collaborations. 

2. Interim Questionnaires 

Each group collaborated to solve the navigation task and within each group 
different people used different styles for collaboration, such as aggressive, passive, 
dismptive, etc.. These differences represent the heart of this research effort. Individual 
personality traits are one factor involved in the exhibition of these differences. Those 
individuals with personality traits associated with emergent leadership (extraverted, 
conscientious, etc.) should display those traits in a physical way (i.e. talkativeness) and 
then emerge as the leader. 

a. Scoring 

The completed interim questionnaires were converted to a database of 
scores associated with each participant. Each participant’s mark for a given measure was 
represented as a value from 0 to 10. The raw data is represented in Appendix C. As each 
group had three members and each survey had five measures, each participant was 
assigned 15 values for each of the three different CVE interfaces uti liz ed. 

3. Post Questionnaires 

After completing the task within each of the three CVE interfaces and the 
associated interim questionnaires, the participants were directed to complete a final 
survey (Appendix B) to elicit general comments regarding the CVE interfaces and group 
collaboration. An additional measure was taken for each group member to evaluate the 
others as to their role as the overall leader of the group. This evaluation was converted to 
a numeric value in the same fashion as the questions on the interim survey. 

B. ANALYSIS OF MEASURED LEADERSHIP SCORES 

Two main avenues of approach were taken for analyzing the data collected from 

the 18 participants. The first approach was to look at the scores in a general way similar 

to the way Slater (2000) measured the emergent leader in his research. The second 

approach entailed a statistical correlation of personality, interface, and emergent 
leadership. 
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1. General Analysis 

In the research conducted by Slater emergent leadership was not the primary 
focus. It was merely an observation of interest that bore notation. The approach used 
was to evaluate the mean and standard deviations for the leadership scores of each 
individual related to the interface they used within the environment. He also analyzed the 
number of times a particular individual had the highest score amongst the other members 
of the same group. 

Taking this approach, this study summed and averaged each of the values from 
the interim questionnaires for each participant. The range of summed values for each 
measurement on the questionnaire was 0 - 30, as there were three people evaluating that 
characteristic. The values for each characteristic were also added to create a range of 
possible values from 0-150 for any one participant at a particular CVE interface. These 
scores were then tabulated according to personality traits and interfaces. Specifically, for 
each individual they were categorized as “predicted leaders” or “non-leaders” according 
to personality traits and those scores across all interfaces were summed and average. 
Additionally, the scores across all the individuals at each of the three different immersion 
workstations were also categorized. The number of times a particular individual’s scores, 
or the scores of the people using a particular interface, was the highest was also recorded. 
Tables 3 and 4 show the relationship of these scores. Some information can be gained 
from these tables, but only in an exploratory way. 



Total Score 

Avg 

Std Dev 

Frequency 

Leader 

1369.5 

273.9 

26.5 

43.0 

Non-leader 1 

1010.4 

202.1 

24.1 

21.0 

Non-leader 2 

1203.6 

240.7 

23.7 

26.0 


Table 3. Mean, Standard Deviation and Frequency of Highest Score per Personality. 
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Total Score 

Avg 

Std Dev 

Frequency 

Cave 

1565.4 

313.1 

20.1 

30.5 

PV230 

1507.1 

301.4 

23.4 

24.5 

Single 

1480.4 

296.1 

19.3 

35.0 


Table 4. Mean, Standard Deviation and Frequency of Fiighest Score per Interface. 

2. Statistical Analysis 

For a more complete analysis, scores for each participant and each immersion 
interface were analyzed using analysis of variance (ANOVA). To analyze the effects the 
interface had on potential emergent leadership, a 1-way ANOVA was conducted with 
three treatments. The box plot of the data (Figure 6) shows the distribution of the scores 
for each interface. The results of the statistical ANOVA show an acceptance of the null 
hypothesis that all the means are equal with a probability of 0.8 (F-test: 2, 51 degrees of 
freedom). A post-hoc power analysis was conducted to recognize possible Type II errors 
- acceptance of null hypothesis that was actually false. Using an a = 0.1 on 3 treatments 
with a population of 18 a low power (1 - (3) = 0.39 was calculated. This indicates there is 
a high probability that a significant difference between treatments did exist but could not 
be detected due to low sample size. 
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Figure 6. Distribution of Emergent Leadership Scores by Interface. 


A slight variation in approach was used for analyzing the effect personality traits 
have on emergent leadership. Although each participant had different personality traits, 
there was only a distinction made between the individual that had the traits related to 
emergent leadership and those that did not have those traits. This distinction required an 
analysis between two groups, leaders and non-leaders, with different population counts. 
The paired Ttest was inappropriate for this dataset, so a 1-way ANOVA was again used 
to account for the different populations. Figure 7 shows the distribution of the scores for 
the two personality categories across all interfaces. The result of this analysis was to 
reject the null hypothesis that the means were equal. The Ftest yields / = 6.84 is greater 
than F(o.o 5 , t, 52 ) = 4.02 (p = 0.012). 
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Figure 7. Distribution of Emergent Leadership Scores by Personality Across All Interfaces. 

One additional analysis was conducted, but did not produce significant results. 
Using the data gained from the post questionnaire, which required the participants to rate 
each other as overall leader, another unbalanced ANOVA was completed. The box plot 
in Figure 8 shows the distribution of data. The results of this analysis were to not reject 
the null hypothesis that the means were equal. The Ftest yields / = 4.37 is not greater 
than F (0 .o5, i, 16) = 4.49 (p = 0.053). 
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Figure 8. Distribution of Overall Leadership Scores by Personality. 

C. ANALYSIS DISCUSSION 

The results obtained from looking at the mean and frequency of selection provides 

some insight into the group collaboration. Table 3 clearly shows that those individuals 

possessing the “emergent leader” traits scored higher, on average, than the other group 

members. It is even more noticeable when examining the frequency for people who 

received the highest leadership characteristics scores. In these cases, the predicted 

leaders were selected nearly 2 to 1 over the other group members. This result was 

co nfir med through statistical analysis using the unbalanced ANOVA with an almost 99 

percent confidence interval. Additionally, results obtained from evaluation of each group 

member as the “overall leader” reveal similar results. The analysis did not produce a 

significant result, but this may be due to the small population size. Those results show 

significance at an alpha level of 0.10, but not at 0.05. 
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The results regarding interface effects differs from the results obtained in Slater’s 
research. The study reported here found some variation between mean leadership scores 
and the number of times these participants were selected as leaders, but does not 
demonstrate any clear relationship between these two variables. The results obtained 
from the ANOVA indicate the mean leadership scores were equal between the different 
interfaces. 

There are several explanations to account for the differences observed. During 
Slater’s (2000) experiment all the group members were required to collaborate on the 
task, but one of the participants was instructed to nonchalantly obstmct the view of one of 
the other participants. This additional duty was also never given to the participant with 
the most immersive interface (i.e. Head Mounted Device). As this additional duty could 
potentially detract from the ability of the person to communicate and collaborate, then 
they would be at a disadvantage. This study did not implement any additional tasks in 
the manner of Slater’s study. So, this could account for some of the difference but not 
all. Another consideration is the use of the HMD. In general, a HMD will provide a 
more immersive interface than the CAVE-like interface used in this study. The physical 
field of view and view orientation attributes of the HMD also produce different 
interactions with the CVE and the real world. The HMD does not easily allow the user to 
switch between real and virtual worlds. Hence, the user tends to become more immersed 
over time. This study required the users to review maps that necessitated switching 
between real and virtual environments to complete the task. This switching could have 
reduced the ability of the various interfaces to fully the immerse the participants in the 
virtual environment. An additional note, the participants in Slater’s study were involved 
in performing the task in the CVE for a greater time period than this study which also 
may have had an effect on the degree of immersion. 

Hie conclusions to be gained from these analyses are that the interface used with 

a CVE, or the level of immersion the interface provides to the user, does not appear to 

significantly affect the emergence of leadership. Further study must be given to this area 

of collaborative virtual environments in order to explain the differences found between 

this research and Slater’s. A larger sample size might have permitted forming groups 

based on more extreme scores which might have produced the anticipated results. 
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Additionally, the personality traits associated with emergent leadership do play a 
significant role in emergent leadership within collaborative virtual environments. 
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V. CONCLUSION AND FUTURE WORK 


A. RESTATEMENT OF OBJECTIVE 

As technologies like high-speed networks, audio and video conferencing, and 
virtual reality improve and gain more acceptance, people will be more involved in 
collaborative work between distant locations. Collaborative virtual environments may 
play a significant role and we must better understand the implications involved in their 
use. This is especially important when these systems are used for training to help ensure 
adverse training effects are avoided. 

This study was designed to whittle away at some of the unknown quantities of 
collaboration within distributed virtual environments. To this end, the study looked 
specifically at the relationship between system interface, personality traits, and emergent 
leadership within a collaborative virtual environment. By using the predictability 
personality traits provide for detecting emergent leadership, this research evaluated who 
emerged as the leader during small group collaboration and compared this with who was 
the predicted leader based on personality traits. As each group member was exposed to 
three different interfaces each with a different level of immersion, any effects the varied 
interface provided to the users should have been uncovered. It was theorized, based on 
previous research (Slater, et al., 2000), that the lim itations of low-end interfaces would 
negatively impact the transfer of leadership personality traits within the virtual 
environment, but not to such a degree that the li mitations could not be overcome. 

B. CONCLUSIONS 

Two main conclusions were formed based on the results of the experiment. The 
first of these conclusions is directly related to emergent leadership personality traits and 
the applicability of these traits within virtual environments. The conclusion states: 

• The personality traits associated with emergent leadership do play a 
significant role in emergent leadership within collaborative virtual 
environments. 


The second conclusion actually argues against the conclusions generated in previous 

research by Slater (2000) at University College, London. Slater’s results indicated that 
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the interface is a major contributing factor to emergent leadership, but the results of this 
research conclude: 

• The interface to a CVE, or the level of immersion the interface presents to 
the user, may not significantly affect the emergence of leadership. 

There may be several possible explanations for the difference in conclusions, and one 
significant point is that Slater’s research did not take the dynamics of personality into 
account while studying the group interaction. 

C. FUTURE WORK 

One point is clear when looking at the results of this research, Slater’s, and others 
lik e it, is that further research is needed to fully understand the effects virtual 
environment technologies have on group collaboration. Specific attention must be paid 
to groups that collaborate within a shared virtual environment when the participants are 
not co-located. 

Many factors are involved in the dynamics of group interactions in addition to 
personality. This research took precautions to ensure the participants within each group 
remained anonymous and were gender homogenous. Other research indicates that 
attributes, whether simple or complex, can have an impact on group dynamics and gender 
is among these attributes (Kimble & McNeese, 1987). Group dynamics become more 
complicated when several factors are considered: group gender mix, leader gender, etc. 
Future research should consider gender and other attributes of group dynamics and how 
different CVE interfaces affect the roles people take within groups. Below is a partial list 
of topics for future work. 

• Emergent leadership in groups with a gender mix 

• Measuring leadership qualities in groups with appointed leaders 

• Effects different interfaces have on training leadership within small groups 

• Evaluating effects CVE interfaces have on things other than leadership 
(aggression, stress resolution, situational awareness, etc.) 
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• CVE interface effects within domains other than land navigation or 
reconnaissance (personnel management, academic research, training, etc.) 

• Comparison between real and virtual world group dynamics for similar 
tasks 

• Qualities required of a CVE to support effective collaboration 

Hie United States military is deeply involved in researching the use of virtual 
environment technologies for training individuals and teams. Virtual environments are a 
logical choice for some training platforms due to the costs and dangers involved in some 
of the training requirements. Before these training systems are implemented for field use, 
a thorough understanding of the effects different CVE interfaces provide must be 
achieved. With a good understanding of these effects the best systems can be employed 
to produce positive training transfer for individuals and groups. 
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APPENDIX A. VIRTUAL MODEL IMPLEMENTATION 


A. HARDWARE CONFIGURATION 

This appendix provides the detailed information on how the hardware and 
software was configured to support this research. 

1. High-Immersion Workstation 

During the planning stage of this research special consideration was given to the 
hardware requirements for this workstation. The research conducted by Slater uti li zed a 
Head-Mounted Display (HMD) for the high-immersion workstation. For similarity 
purposes, a HMD type system was desired, but the collaborative task made using a HMD 
impractical. The task required the user to interact with “real-world” tools like maps in 
order to achieve the objectives. The resolution capabilities of current HMD technologies 
are not sufficient to support detailed map displays. Thus, a CAVE-l ik e system was 
ut ili zed to provide a highly immersive interface to the user. 
a. Display Hardware 

The display system utilized a Multi-Angle Virtual Environment (MAVE) 
developed from previous research at NPS (Figure 9). The MAVE consisted of three large 
polarization-preserving rear-projection screens that were each 7 feet wide and 6 feet high. 
Each of these screens was raised above the floor 22 inches to provide a more natural 
viewing angle to a participant in a standing posture. For this experiment the participants 
were provided an elevated stool to reduce fatigue and facilitate an eye position 
approximating a standing posture. The three displays were placed at 45 degree angles to 
each other and produced a physical field of view of approximately 135 degrees when the 
user was at the optimal viewing distance from the center display of approximately 6 feet. 
Each display had a dedicated VRex VR2210 polarized stereo LCD projector that 
projected the image onto the screens via two reflecting mirrors. The three displays 
existed within a small room with sound absorbent surfaces and no external distraction. 
Overall this provided a highly immersive environment for the experiment participants. 
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Figure 9. User at High-Immersion Workstation. 
b. Computer Hardware 

This workstation required the highest level of computational power. This 
was achieved by building three machines dedicated to rendering the CVE images to the 
three projectors. Each machine was produced from commercial off-the-shelf products as 
indicated in Table 1. The graphical rendering synchronization between the machines was 
controlled by software over a dedicated high-speed 1000 Megahertz. Overall, the 
computer configuration smoothly reproduced the CVE with average frame rates greater 
than 25 frames per second at a resolution of 1024 by 768 pixels. 

2. Mid- Im mersion Workstation 

The basic requirements for this workstation were to provide the user an interface 
to Ihe CVE that was more immersive than a single-screen display and less than that of the 
high-immersion workstation. 

a. Display Hardware 

The display system utilized a specialized panoramic display (PV230) 
developed by Panoram Technologies (Figure 10). The display consisted of three mid¬ 
sized flat panel LCD screens connected together at approximately 25 degree angles. The 
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seams between each display were approximately 11 millim eters and produced little 
distraction when the user’s focal point transitioned between the displays. Each individual 
display had a 15 inch diagonal measurement and all combined provided a 36 inch wide 
by 9 inch high viewing area. Overall this provided a pleasing, high quality display with a 
wide field of view, but not as immersive a display as the high-immersion workstation. 



Figure 10. User at Mid-Immersion Workstation. 
b. Computer Hardware 

This workstation utilized a high-end Dell personal computer augmented to 
support multiple displays as indicated in Table 2. Due to the complexities of the virtual 
model, the computer configuration reproduced the CVE with a disappointing average 
frame rate of approximately 10 frames per second at a resolution of 1024 by 768 pixels. 
Although this frame rate could have been improved by reducing the image resolution, it 
was considered more important to ensure each workstation had identical pixel 
resolutions. The frame rate difference was noticeable, but participant post-experiment 
comments regarding frame rate did not reveal any significant display problems. It is 
theorized the slow movement associated with “walking” in the CVE allowed the users to 
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cope with the low frame rate. The difference in frame rate was most noticeable while 
changing view azim uth or elevation. 

3. Low-Immersion Workstation 

The basic requirement for this workstation was to provide the user an interface to 
the CVE that had few immersive qualities, as found in typical personal computer 
systems. 

a. Display Hardware 

The display system utilized a single 21 inch diagonal monitor as part of a 
multiple display system; the additional displays were not utilized for the experiment. 
Although the diagonal measurement of this display was greater than the mid-immersion 
workstation, the total viewing area was less and covered a much smaller physical field of 
view. Overall, this interface provided the least capability to immerse the participant in 
the CVE. 



Figure 11. User at Low- Immersion Workstation. 
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b. Computer Hardware 

This workstation ut ili zed a high-end Intergraph TDZ 2000 personal 
computer augmented to support multiple displays as indicated in Table 2. Overall, the 
computer configuration reproduced the CVE with an average frame rate of approximately 
20 frames per second at a resolution of 1024 by 768 pixels. 

B. SOFTWARE CONFIGURATION 

All the CVE workstations utilized the same software with minor variations to 
support the different interface requirements. The real-time simulation software ut ili zed 
for the CVE was Multigen-Paradigm’s Vega software. Several modules were attached to 
the software kernel to support the requirements of the experiment. The list below 
ident if ies the main modules of concern and their use requirement. 

• ‘ ‘DI- Guy' ’ for avatar implementation 

• “DIS/HLA" for network protocol support 

• “Symbology" for implementation of a compass and distance display 

• “Distributed Vega” for multi-machine multi-pipe scene rendering 

All the modules were available at every workstation, but the symbology and distributed 
Vega modules were only activated on the computer systems that required that 
implementation. Specifically, the Distributed Vega module was only activated for the 
high-immersion workstation and Symbology was only implemented for users that had the 
compass as part of their tool set. 

One complication that occurred during this experiment was that not all the 
modules functioned together properly. An exorbitant amount of time was spent 
configuring the software to adequately support effective user control and display of an 
avatar within the networked virtual environment. The DIS module makes sharing 
environments and virtual objects across networks very simple to implement. 
Additionally, the DI-Guy module is a great way to introduce humanoid objects into a 
CVE, but they were designed for scripted behaviors. Another complication was that the 
DI-Guy is represented within the Vega software as a special kin d of object. As a special 
object, the avatar is not sent across networks by the DIS module in the same way as other 
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objects. In essence, these peculiarities of the software complicated the development 
process. The next sections document some of the solutions implemented to support this 
research. Specifically, the code implementation to allow a user to control an avatar and 
then represent the state of that avatar at the receiving workstations will be described. 

1. Avatar Control 

The DI-Guy software module was developed to support the introduction of 
humanoid objects with realistic human behaviors into real-time simulations. These 
objectives have been met quite effectively, but are done using pre-scripted behaviors and 
movement paths. There are hooks available to allow software to dynamically control the 
actions of the avatar, but when these are utilized the realism of the behaviors and actions 
is reduced. 

The requirements of this study included the ability of the user to point at objects 
or in a given direction. To support this, the avatar required direct user control and all the 
group members needed to see the pointing behavior. Control was achieved through 
keyboard actions that translated to a change of avatar “state”. The code fragment below 
(Figure 12) illustrates how those keyboard commands were utilized. When a keyboard 
event was captured the event information was parsed to determine which key was pressed 
and then the appropriate avatar state was set and that state was transmitted over the 
network to the workstations. An additional adjustment was also made to the viewing 
height of the user dependent on whether the avatar state changed from standing to 
kneeling. 
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switch(keyPressed) { 

//******* the following key events are for DI-Guy motions ****** 
case VGWIN_UPARROW: //character walk (property value = 6) 
case VGWIN_PAD8: 

//****** change avatar state 

vgDIGuyAction (vgCharHandle, SOLDIER_WALK); 

//****** adjust observer view point 
chgObsOf f setAlt(SOLDIER_WALK) ; 

//****** send State change info to DIS 

sendState(SOLDIER_WALK); 

break; 

case VGWIN_PAD9: //character jog (property value = 11) 
case 'r' : 
case 'R': 

//****** change avatar state 
vgDIGuyAction (vgCharHandle, SOLDIER_JOG); 

//****** adjust observer view point 
chgObsOf f setAlt(SOLDIER_JOG) ; 

//****** send State change info to DIS 

sendState(SOLDIER_JOG); 

break; 


Figure 12. Code to Control DI-Guy Avatar State. 


The pre-scripted behaviors for which the DI-Guy object was designed ut ili zed 
transitions between states to achieve the realistic human actions when switching avatar 
states. For instance, if the avatar switched from running to standing, then the script 
would actually adjust the avatar from a running state to a walking state and then a 
standing state. This created a visually realistic state transition. When sending state 
information about the avatar over the network these intermediate states must be sent as 
well. For simplicity reasons, these intermediate states were not transmitted which could 
cause the avatar on the receiving workstation to get stuck in a particular state. To remedy 
this feature, the state transitions were disabled and the avatar immediately changed state. 
This created some jerky actions when transitioning between certain avatar states. 

2. Sending and Receiving Avatar State Information 

The DIS network module was designed to automatically send state and positional 
information about objects to all the nodes in the shared CVE. Unfortunately, the DI-Guy 
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is treated as a special type of object and this information is not captured and sent properly 
when the avatar is under user control. Thus, the state information of the avatar was 
manually sent to the other nodes on the network. This was implemented via a separate 
multicast channel established between all the workstations. Sending the information was 
simple and straightforward; the complications occurred on the receive side. 

Each workstation had an established identification number within the CVE and 
this number attached to the avatar state information that was sent to the other 
workstations. Upon receipt, the receiving node would parse the network packet for the 
station identification and then locate the associated avatar that needed to be adjusted. 
Typically, this locating of the associated avatar would be a trivial search for the object 
name and assigning a pointer, but because of the special nature of the DI-Guy objects this 
was complicated. The DI-Guy objects that represent another workstation user are 
dynamically created and destroyed during the simulation, and each time this happens a 
new name is assigned to the object. Thus, the name of the object is never known. To get 
around this problem, each user’s avatar was assigned a specific name and that name was 
linked to the workstation identification number. Then, the receiving workstation could 
identify each avatar by the link between the name and the number. 

3. Adjusting Avatar Position 

Another compatibility problem between the modules appeared in relation to the 
position of the avatar on the receiving station. The DIS module automatically reported 
the position information of an object within the CVE. For DI-Guy objects, the module 
does not send the position information of the avatar, but sends the position of another 
object that represents the avatar. When the positional information is received at a 
workstation the DIS module automatically decodes the information and sets the position 
of another object that represents the avatar rather than the avatar itself. This 
representative object exists because of the special nature of DI-Guy. The action states 
and behaviors associated with the DI-Guy avatar fend to make the avatar position on the 
receiving station shift in relation to the representative object position. To fix this 
problem the avatar object position needs to be periodically adjusted back to the position 
of the representative object. Figure B shows the function used to make this adjustment. 
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This drift and adjustment occur for every instance of networked DI-Guys, so the function 
simply adjusts all the inbound avatar positions. 


void diGuyCorrection () { 

vgDIGuy *d = NULL; 
float x=0.0f, y=0.0f, z=0.0f; 

bdiCharacterHandle bdiChar = NULL; 

//loop through all the DIS DI-Guys and 
//find the active DIS DIGuy entities 

for (int index = 0; index < MAX_DIS_DIGUYS; index++) { 
//check if array holder has a valid DIS entity 
if (disDIGuyNames[index] != NULL) { 

//try to find the associated DIGuy 

if ( (d = vgFindDIGuy(disDIGuyNames[index])) != NULL) ( 

//get the associated BDI Handle 
bdiChar = vgGetDIGuyCharacterHandle(d); 

//teleport to 0,0,0 

diguy_teleport(bdiChar, &x, &y, &z); 

}//end if 
}//end if 
}//end for loop 
(//end diGuyCorrection 


Figure 13. Code to Adjust DI-Guy Avatar Position. 


There were many adjustments made to the experiment model to create 
controllable avatars that could be used by the participants in a way that was conducive to 
the navigation task. Although not all the problems were fixed, the participants did not 
report an inability to achieve the task objectives based on avatar control and 
representation. Most users enjoyed the experience and found the pointing, along with 
other features, important to achieving the objectives. 
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APPENDIX B. EXPERIMENT FORMS 


A. GENERAL 

Hie forms in this appendix appear in the same format uti liz ed for the experiment 
and do not follow the standard thesis formats ut ili zed in the chapters of this document. 
The appendix consists of six documents: Privacy Act Statement, Participant Consent 
Form, Minimal Risk Consent Statement, In-brief Statement, Interim Questionnaire, and 
the Post Questionnaire. 

1. Privacy Act Statement 

The following page represents the Privacy Act Statement presented to each 
participant. Each participant was required to read and sign this document prior to 
participating in the experiment. 
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PRIVACY ACT STATMENT 


NAVAL POSTGRADUATE SCHOOL, MONTEREY, CA 93943 
PRIVACY ACT STATEMENT 

1. Authority: Naval In struction 

2. Purpose: Personality inf ormation will be collected to enhance knowledge, and to 

develop tests, procedures, and equipment to improve the development of Distributed 
Collaborative Virtual Environments. 

3. Use: Personality information will be used for statistical analysis by the Departments 
of the Navy and Defense, and other U.S. Government agencies, provided this use is 
compatible with the purpose for which the information was collected. Use of the 
information may be granted to legitimate non-government agencies or individuals by 
the Naval Postgraduate School in accordance with the provisions of the Freedom of 
Information Act. 

4. Disclosure/Confidentiality: 

a. I have been assured that my privacy will be safeguarded. I will be assigned a 
control or code number, which thereafter will be the only identifying entry on 
any of the research records. The Principal Investigator will maintain the cross- 
reference between name and control number. It will be decoded only when 
beneficial to me or if some circumstances, which is not apparent at this time, 
would make it clear that decoding would enhance the value of the research data. 
In all cases, the provisions of the Privacy Act Statement will be honored. 

b. I understand that a record of the in formation contained in this Consent Statement 
or derived from the experiment described herein will be retained permanently at 
the Naval Postgraduate School or by higher authority. I voluntarily agree to its 
disclosure to agencies or individuals indicated in paragraph 3 and I have been 
informed that failure to agree to such disclosure may negate the purpose for 
which the experiment was conducted. 

c. I also understand that disclosure of the requested information, including my 
Social Security Number, is voluntary. 


Signature of Volunteer Name, Grade/Rank (if applicable) DOB SSN Date 


Signature of Principal Investigator Date 
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2. Participant Consent Form 

The following page represents the Participant Consent Form presented to each 
participant. Each participant was required to read and sign this document prior to 
participating in the experiment. 
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PARTICIPANT CONSENT FORM 


1. Introduction. You are invited to participate in a study of group interactions within 
virtual environments. With information gathered from you and other participants, we 
hope to discover insight on communication techniques used while collaborating within 
a virtual environment during dismounted navigation of natural terrain. We ask you to 
read and sign this form indicating that you agree to be in the study. Please ask any 
questions you may have before signing. 

2. Background Information. The Naval Postgraduate School NPSNET Research Group 
is conducting this study. 

3. Procedures. If you agree to participate in this study, the researcher will explain the 
tasks in detail. There will be two sessions: a) 30 minute pretest phase and b) 60 minute 
training and execution phases, during which you will be expected to accomplish a 
number of tasks related to navigating a natural terrain environment. 

4. Risks and Benefits. This research involves no risks or discomforts greater then those 
encountered in ordinary computer usage. The benefits to the participants are gaining 
techniques for enhancing spatial knowledge of unfamiliar environments and 
contributing to current research in human-computer interaction. 

5. Compensation. No tangible reward will be given. A copy of the results will be 
available to you at the conclusion of the experiment. 

6. Confidentiality. The records of this study will be kept confidential. No information 
will be publicly accessible which could identify you as a participant. 

7. Voluntary Nature of the Study. If you agree to participate, you are free to withdraw 
from the study at any time without prejudice. You will be provided a copy of this form 
for your records. 

8. Points of Contact. If you have any further questions or comments after the completion 
of the study, you may contact the research supervisor, Dr. Rudolph P. Darken (831) 
656-4072, darken@nps.navy.mil. 

9. Statement of Consent. I have read the above information. I have asked all questions 
and have had my questions answered. I agree to participate in this study. 


Signature of Volunteer 


Date 


Signature of Principal Investigator Date 
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3. 


Minimal Risk Consent Statement 


The following page represents the Minimal Risk Consent Statement presented to 
each participant. Each participant was required to read and sign this document prior to 
participating in the experiment. 
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MINIMAL RISK CONSENT STATEMENT 


NAVAL POSTGRADUATE SCHOOL, MONTEREY, CA 93943 
MINIMAL RISK CONSENT STATEMENT 

Participant: VOLUNTARY CONSENT TO BE A RESEARCH PARTICIPANT IN: 

Virtual Environment Interface Effects on Collaboration 

1. I have read, understand and been provided "Information for Participants" that provides the 
details of the below acknowledgments. 

2. I understand that this project involves research. Ai explanation of the purposes of the 
research, a description of procedures to be used, identification of experimental procedures, 
and the extended duration of my participation have been provided to me. 

3. I understand that this project does not involve more than minimal risk. I have been i nf ormed 
of any reasonably foreseeable risks or discomforts to me. 

4. I have been informed of any benefits to me or to others that may reasonably be expected from 
the research. 

5. I have signed a statement describing the extent to which confidentiality of records identifying 
me will be maintained. 

6. I have been i nf ormed of any compensation and/or medical treatments available if injury 
occurs and is so, what they consist of, or where further information may be obtained. 

7. I understand that my participation in this project is voluntary, refusal to participate will 
involve no penalty or loss of benefits to which I am otherwise entitled. I also understand that 
I may discontinue participation at any time without penalty or loss of benefits to which I am 
otherwise entitled. 

8. I understand that the individual to contact should I need answers to pertinent questions about 
the research is Krist Norlander, Principal Investigator, and about my rights as a research 
participant or concerning a research related injury is the Modeling Virtual E nvironments and 
Simulation Chairman. A full and responsive discussion of the elements of this project and 
my consent has taken place. 

Medical Monitor: Flight Surgeon, Naval Postgraduate School 


Signature of Volunteer Date 


Signature of Principal Investigator Date 
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4. 


In-brief Statement 


The following page represents the In-brief Statement presented to each 
participant. Etch participant was required to read this document prior to commencing the 
experiment. 
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Collaboration In-Brief 


You, as a participant in this experiment, have been assigned to a small collaboration group consisting of 
three (3) persons. Your assignment to this group was predicated on the results obtained from the 
personality questionnaire completed in the first stage of the experiment. 

Experiment Focus: 

The focus of the experiment is on collaboration interaction within a small group. For data collection 
purposes your participation will be recorded on video. 

Objective: 

Your objective in this experiment is to work with the other participants in your group to search, identify, 
and locate as many targets as possible within the time period allotted. This task is similar to a military 
reconnaissance mission. 

Tools: 

Each person in your group has a different set of tools available to assist in achieving your goal. The tools 
available to any one participant are not enough to effectively solve the problem. The best results will be 
achieved through collaboration with the other group members. It will be advantageous to the group for 
each member to be aware of the tools the other members have available. 

Collaboration: 

You can talk to the other participants via the audio headset provided. This headset includes a voice- 
activated microphone that supports full-duplex network communication. 

In addition to the headset, each person in your group will be represented by an avatar (computer generated 
soldier). The other participants will see your avatar and you will be able to see theirs. Your avatar has a 
limited set of interactions that will represent what you are doing in the virtual environment. The avatar 
can “stand”, “walk”, “run”, and “aim a rifle”. The avatar state of “aiming your rifle” will be used to 
show the other participants that you are pointing in a given direction. 

Requirements: 

Throughout the experiment you will be required to identify the other group members by one of the 
following names: “Echo”, “Kilo”, “Tango”. Even if you think you know another group member via 
voice recognition, you are required not to address that person by their real name. 

The only other requirement is that all members of the group must remain together and agree on the 
actions of the group. Each member will have a diagram for marking the position of the targets located 
and identified. 

Experiment Sequence: 

You will participate in three (3) 15-minute reconnaissance sessions. After each session, you will be 
required to complete a short series of questions about the roles each member undertook within the virtual 
environment and then rotate to a new interaction station. 

Practice: 

When you are ready, you will be provided a five (5) minute period to become familiar with the navigation 
procedures within the virtual environment. A short task list will be provided to assist in familiarization. 

Please, ask any questions you have prior to starting the experiment. During the experiment, the Principle 
Investigator will monitor the audio channel and be available to answer a limited set of questions. 
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5. Interim Questionnaire 

Hie following page represents the Participant Interim-Questionnaire presented to 
each participant during the experiment. Each participant was required to complete this 
survey after completing the navigation task at each workstation. 
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Participant ID:_ Callsign:_ Run #_ 

Participant Interim Questionnaire 

Please, answer the following questions as honestly as possible. Your responses will be utilized to better understand 
the interactions of each member of this collaboration group exercise. All questions should be answered referencing 
the collaboration experience you iust completed . Mark your responses on the scale by placing an “X” at the 


appropriate level. 

Least |—.|-1.|-1.1—~^-i-1—.|-1-1 Most , 

Identify how much of the overall talking Was done by each group member (including yourself). 

ECHO Least |—.|-1-1.|-1-1-1—.|-1-1 Most 

KILO Least |.|.|-1—.|.|.|-1-1-1-1 Most 

TANGO Least |.|.|-1.|-1-1.|-1-1-1 Most 

Identify the amount of verbal feedback (response to another member) that was provided by each group member 
(including yourself). 

ECHO Least |-1-1-1.|-1-1-1.|-1-1 Most 

KILO Least |-1-1-1-1.|-1-1.|-1-1 Most 

TANGO Least |-1-1—.|.|-1-1-1-1-1-1 Most 

Identify the quality (“helpfulness”) of the verbal feedback that was provided by each group member (including 
yourself). 

ECHO Low |.|-1.|-1-1-1-1-1-1-1 High 

KILO Low |-1-1-1-1.|-1.|-1-1-1 High 

TANGO Low |-1-1.|.|.|.-|-1-1-1.| High 

Identify the amount of input that was provided by each group member (including yourself) that appeared effective at 
achieving the objectives (communicating, finding targets, and positioning targets on maps). 

ECHO Least |-1.|.-|-1-1-1-1-1.|-.| Most 

KILO Least |-1-1.-|-1.|-1—-.|-1-1-1 Most 

TANGO Least |-1—.|-1-.|-1-1-1-1-1-1 Most 

Identify the amount of overall involvement by each group member in discussions and decisions. 

ECHO Least |-1-1-1-1.|.|.|-1.|-1 Most 

KILO Least |.|.|———|.|.|-1-1-1-1-1 Most 

TANGO Least |-1-1.|.|.|-1.|-1-1-1 Most 
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6. Post Questionnaire 

The following page presents the Participant Post-Questionnaire presented to each 
participant after completing the experiment. Each participant was required to complete 
this survey after completing the navigation tasks at all of the three workstations. 
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Participant ID: 


Participant Post Questionnaire 

Please, answer the following questions as fully and honestly as possible. Your responses will be utilized to improve 
future research in Distributed Collaboration. When answering a question regarding interface, each interface should 
be identified by the following terms: 

CAVE - Large 3 screen display 

PV230 - Small 3 screen display 

Single - Small single screen display 


1. Which interface provided you the most enjoyable experience? Why? 


2. What was your preferred interface? Why? 


3. Which, if any, of the interfaces did you feel interfered the most with your ability to meet the objectives 
(communicate, navigate, locate and identify targets)? 


4. If applicable, how did the interface identified in above (question 3) interfere? 


5. Which, if any, of the interfaces made you feel “less a part” of the group? Why? 


6. Did you experience difficulties verbally communicating yourself over the networked communication system? 

7. If applicable, describe the verbal communication conflicts experienced? Why? 


8. Did you find the “pointing” capability useful to group interaction? Why or why not? 


9. Overall, which group member would you identify as the leader ? Why? 

ECHO Least |-1.|.|-1.|-1-1-1-1-1 Most 

KILO Least |.|.|-1-1-1.|-1.|-1-1 Most 

TANGO Least |-1-1-1.|-1-1-1-1-1--1 Most 
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APPENDIX C. RAW DATA 


A. GROUP 1 - INTERIM- AND POST-QUESTIONNAIRE SCORES 


Participant ID 

16 

53 

59 ! 

Emergent Leader Traits 

No 

Yes 

No 

Run # 

1 

2 

3 

1 

2 

3 

1 

2 

3 

Workstation Interface 

Cave 

Single 

PV230 

PV230 

Cave 

Single 

Single 

PV230 

Cave 

Tool Set 

A 

B 

C 

C 

A 

B 

B 

C 

A 

Amount of Talking 










16 

5.5 

1.6 

9.4 

9.5 

9.5 

7.5 

5.5 

3.4 

7.5 

53 

2.6 

4.4 

5.5 

6.5 

6.5 

5.5 

5.4 

6.5 

6.4 

59 

2.4 

2.4 

3.5 

6.5 

5.6 

4.5 

4.6 

6.4 

4.5 

Feedback Quantity 




16 

3.4 

2.5 

9.5 

3.5 

9.5 

5.5 

3.5 

2.5 

3.4 

53 

3.6 

4.5 

5.5 

4.5 

5.6 

5.5 

6.5 

6.4 

5.5 

59 

1.5 

1.5 

2.5 

7.5 

6.4 

4.5 

6.5 

6.5 

5.4 

Feedback Quality 




16 

3.5 

4.4 

8.3 

8.5 

9.3 

7.6 

4.4 

5.4 

6.6 

53 

0.5 

4.5 

5.5 

0.5 

6.5 

6.4 

0.5 

6.5 

6.4 

59 

0.5 

1.5 

1.5 

7.5 

6.4 

4.5 

4.5 

7.4 

4.5 

Effective Input 




16 

6.5 

5.7 

8.5 

8.6 

6.4 

6.3 

6.5 

7.5 

5.4 

53 

2.4 

4.5 

5.5 

6.4 

5.5 

5.5 

5.6 

6.4 

5.5 

59 

0.5 

1.5 

2.5 

4.5 

5.5 

4.5 

4.5 

5.5 

4.5 

Overall Involvment 




16 

5.6 

1.5 

8.4 

8.6 

7.6 

7.6 

5.6 

3.5 

5.5 

53 

3.4 

4.5 

5.5 

6.4 

5.5 

5.5 

6.4 

5.5 

5.5 

59 

1.5 

0.6 

4.5 

7.3 

5.4 

5.5 

6.5 

7.5 

5.5 


Overall Leader 

16 

53 

59 

16 

5.6 

8.4 

5.5 

53 

3.5 

6.4 

7.4 

59 

2.5 

8.6 

7.5 


Toolset 

Elements 

A 

Compass and Satellite Image 

Key Feature Location with Photos 
Compass Rose and Contour Map 

B 

C 
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B 


GROUP 2 - INTERIM- AND POST-QUESTIONNAIRE SCORES 


Participant ID 

54 

40 

34 

Emergent Leader Traits 

Yes 

No 

No 

Run # 

1 

2 

3 

1 

2 

3 

1 

2 

3 

Workstation Interface 

Cave 

PV230 

Single 

PV230 

Single 

Cave 

Single 

Cave 

PV230 

Tool Set 

C 

A 

B 

A 

B 

C 

B 

C 

A 

Amount of Talking 










54 

4.6 

5.0 

5.0 

4.6 

5.0 

5.0 

4.6 

5.0 

5.0 

40 

5.6 

6.6 

7.4 

6.6 

8.5 

8.4 

7.4 

7.5 

7.4 

34 

5.0 

8.0 

8.0 

7.0 

8.0 

8.0 

3.0 

8.0 

4.0 

Feedback Quantity 




54 

4.7 

5.0 

5.0 

4.7 

5.0 

5.0 

4.7 

5.0 

5.0 

40 

5.4 

7.4 

7.5 

5.4 

8.5 

8.4 

5.4 

7.3 

7.4 

34 

7.0 

8.0 

8.0 

7.0 

8.0 

8.0 

6.0 

7.0 

8.0 

Feedback Quality 




54 

5.1 

5.0 

5.0 

2.9 

5.0 

5.0 

2.7 

5.0 

5.0 

40 

6.5 

6.5 

7.5 

6.5 

7.5 

8.5 

6.5 

8.4 

7.3 f 

34 

7.0 

8.0 

8.0 

7.0 

8.0 

7.0 

7.0 

8.0 

7.0 

Effective Input 




54 

3.6 

5.0 

5.0 

2.6 

5.0 

5.0 

3.8 

5.0 

5.0 

40 

4.4 

6.3 

4.5 

4.4 

8.5 

4.5 

4.4 

7.4 

4.5 

34 

9.0 

9.0 

7.0 

9.0 

9.0 

7.0 

9.0 

8.0 

7.0 

Overall Involvment 




54 

5.0 

5.0 

5.0 

5.0 

5.0 

5.0 

5.0 

5.0 

5.0 

40 

6.5 

7.4 

7.5 

6.5 

8.4 

7.5 

6.5 

7.4 

7.5 

34 

9.0 

9.0 

9.0 

9.0 

9.0 

9.0 

9.0 

9.0 

9.0 


Overall Leader 

54 

40 

34 

54 

5.0 

5.0 

5.0 

40 

5.50 

6.50 

7.50 

34 

10.00 

9.00 

8.00 


Toolset 

Elements 

A 

Compass and Satellite Image 

Key Feature Location with Photos 
Compass Rose and Contour Map 

B 

C 
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c 


GROUP 3 - INTERIM- AND POST-QUESTIONNAIRE SCORES 


Participant ID 

35 

91 

41 

Emergent Leader Traits 

No 

Yes 

No 

Run # 

1 

2 

3 

1 

2 

3 

1 

2 

3 i 

Workstation Interface 

Cave 

Single 

PV230 

PV230 

Cave 

Single 

Single 

PV230 

Cave 

Tool Set 

B 

A 

C 

C 

B 

A 

A 

C 

B 

Amount of Talking 










35 

4.4 

5.3 

4.5 

7.5 

8.4 

7.5 

2.5 

0.6 

6.4 

91 

4.4 

4.5 

4.5 

6.2 

4.5 

5.3 

2.3 

0.0 

6.4 

41 

5.5 

6.5 

3.5 

9.3 

9.5 

6.5 

2.3 

5.6 

7.6 

Feedback Quantity 




35 

7.4 

7.5 

4.5 

3.5 

7.5 

6.5 

6.5 

0.4 

7.4 

91 

7.2 

4.5 

5.5 

1.6 

4.5 

4.5 

3.4 

0.0 

6.5 

41 

4.3 

5.5 

4.6 

3.5 

4.5 

5.5 

6.5 

6.3 

6.7 

Feedback Quality 




35 

7.4 

7.6 

4.4 

5.4 

6.5 

6.5 

1.4 

0.5 

8.4 

91 

8.5 

4.5 

4.5 

8.5 

4.5 

6.5 

2.5 

0.0 

6.5 

41 

6.5 

5.5 

4.5 

7.3 

7.4 

7.6 

3.5 

3.5 

7.7 

Effective Input 




35 

5.5 

7.5 

5.6 

6.6 

6.5 

5.5 

1.5 

0.5 

8.5 

91 

5.5 

4.5 

4.5 

7.5 

4.5 

6.5 

2.5 

0.0 

6.5 

41 

8.5 

5.5 

3.5 

6.5 

8.3 

7.6 

3.5 

2.5 

8.4 

Overall Involvment 




35 

5.4 

7.5 

5.4 

7.5 

7.6 

7.6 

1.4 

0.5 

5.4 

91 

7.5 

4.5 

6.6 

7.5 

4.5 

6.5 

3.5 

0.0 

5.5 

41 

7.3 

6.5 

4.5 

8.4 

7.5 

6.5 

3.4 

3.5 

8.3 


Overall Leader 

35 

91 

41 

35 

5.3 

7.4 

3.4 

91 

0.0 

6.5 

0.0 

41 

0.0 

10.0 

0.0 


Toolset 

Elements 

A 

Compass and Satellite Image 

Key Feature Location with Photos 
Compass Rose and Contour Map 

B 

C 


65 









































































D 


GROUP 4 - INTERIM- AND POST-QUESTIONNAIRE SCORES 


Participant ID 

74 

81 

65 

Emergent Leader Traits 

No 

No 

Yes 

Run # 

1 

2 

3 

1 

2 

3 

1 

2 

3 1 

Workstation Interface 

Cave 

Single 

PV230 

PV230 

Cave 

Single 

Single 

PV230 

Cave 

Tool Set 

A 

C 

B 

B 

A 

C 

C 

B 

A 

Amount of Talking 










74 

0.5 

0.5 

3.5 

8.5 

9.5 

9.3 

8.5 

9.5 

9.3 

81 

2.5 

1.4 

1.0 

8.0 

6.8 

9.0 

6.0 

7.6 

7.5 

65 

4.4 

2.5 

3.4 

9.5 

8.3 

7.6 

7.5 

8.2 

8.5 

Feedback Quantity 




74 

2.5 

3.5 

9.5 

7.4 

5.5 

9.4 

7.4 

5.5 

9.4 

81 

2.6 

2.1 

2.5 

4.8 

7.5 

7.7 

6.8 

7.5 

7.7 

65 

6.3 

1.4 

4.4 

7.4 

7.4 

7.4 

7.4 

7.5 

7.4 

Feedback Quality 




74 

6.5 

0.4 

7.5 

7.5 

9.4 

7.5 

7.5 

9.4 

7.5 

81 

2.8 

6.5 

7.7 

7.4 

7.3 

7.7 

8.3 

9.4 

8.6 

65 

2.5 

0.5 

5.4 

2.5 

5.3 

6.5 

2.4 

5.3 

6.4 

Effective Input 




74 

7.5 

0.6 

7.5 

7.5 

5.5 

6.4 

7.5 

5.5 

6.4 

81 

1.9 

3.6 

3.3 

8.0 

8.3 

7.6 

8.2 

7.3 

7.5 ! 

65 

3.4 

0.6 

4.6 

4.4 

5.6 

6.4 

4.4 

7.3 

6.3 

Overall Involvment 




74 

0.5 

0.5 

2.4 

9.4 

9.5 

9.4 

9.4 

9.5 

9.4 ! 

81 

3.4 

0.4 

0.3 

9.8 

6.5 

7.3 

8.3 

7.9 

7.2 

65 

3.5 

2.4 

4.5 

8.5 

7.5 

7.5 

6.5 

8.5 

8.4 


Overall Leader 

74 

81 

65 

74 

0.6 

9.4 

8.5 

81 

0.4 

8.0 

6.7 

65 

4.3 

7.5 

8.5 


Toolset 

Elements 

A 

Compass and Satellite Image 

Key Feature Location with Photos 
Compass Rose and Contour Map 

B 

C 


66 









































































E. 


GROUP 5 - INTERIM- AND POST-QUESTIONNAIRE SCORES 


Participant ID 

92 

85 

77 

Emergent Leader Traits 

No 

No 

Yes 

Run # 

1 

2 

3 

1 

2 

3 

1 

2 

3 i 

Workstation Interface 

Cave 

Single 

PV230 

PV230 

Cave 

Single 

Single 

PV230 

Cave 

Tool Set 

C 

B 

A 

A 

C 

B 

B 

A 

C 

Amount of Talking 










92 

5.5 

4.0 

2.0 

5.5 

4.0 

2.9 

5.5 

4.0 

3.1 

85 

7.5 

6.5 

6.4 

3.5 

6.5 

7.5 

6.5 

6.5 

5.6 

77 

4.8 

5.4 

5.2 

3.6 

5.4 

5.2 

6.4 

5.0 

5.8 

Feedback Quantity 




92 

5.0 

2.0 

3.1 

5.0 

2.9 

3.0 

5.0 

3.1 

3.1 

85 

6.4 

6.5 

4.6 

6.5 

2.4 

7.5 

6.6 

5.4 

5.6 

77 

4.8 

5.9 

5.7 

4.8 

5.0 

5.8 

4.9 

5.0 

5.2 

Feedback Quality 




92 

1.0 

1.0 

3.0 

1.0 

1.0 

3.0 

1.0 

1.0 

3.1 

85 

7.5 

3.5 

7.3 

6.5 

3.5 

8.6 

5.5 

3.5 

5.5 

77 

5.8 

5.8 

5.4 

5.8 

4.5 

5.3 

5.8 

4.1 

5.7 

Effective Input 




92 

3.0 

2.0 

3.0 

3.0 

2.0 

3.0 

3.0 

2.0 

3.0 

85 

5.5 

4.5 

6.3 

2.6 

4.5 

7.6 

5.5 

4.5 

5.5 i 

77 

5.1 

5.3 

5.8 

5.0 

4.8 

5.8 

5.0 

4.2 

5.8 

Overall Involvment 




92 

5.0 

3.0 

2.0 

5.0 

3.9 

2.9 

5.0 

4.0 

3.1 

85 

6.6 

7.5 

6.4 

6.5 

7.6 

7.5 

6.6 

7.6 

6.4 1 

77 

5.0 

5.2 

5.5 

5.0 

5.2 

4.6 

5.7 

5.3 

5.3 


Overall Leader 

92 

85 

77 

92 

3.0 

3.0 

3.0 

85 

4.5 

4.5 

4.5 

77 

5.3 

5.2 

5.3 


Toolset 

Elements 

A 

Compass and Satellite Image 

Key Feature Location with Photos 
Compass Rose and Contour Map 

B 

C 


67 









































































F. 


GROUP 6 - INTERIM- AND POST-QUESTIONNAIRE SCORES 


Participant ID 

52 

64 

48 

Emergent Leader Traits 

No 

Yes 

No 

Run # 

1 

2 

3 

1 

2 

3 

1 

2 

3 1 

Workstation Interface 

Cave 

Single 

PV230 

PV230 

Cave 

Single 

Single 

PV230 

Cave 

Tool Set 

B 

A 

C 

C 

B 

A 

A 

C 

B 

Amount of Talking 










52 

2.0 

4.7 

4.6 

3.3 

5.4 

6.5 

5.2 

4.5 

4.5 

64 

3.0 

3.0 

4.0 

8.0 

10.0 

9.0 

8.0 

8.0 

9.0 

48 

5.0 

4.0 

6.0 

6.0 

6.0 

6.0 

7.0 

5.0 

6.0 

Feedback Quantity 




52 

2.8 

6.2 

5.8 

3.5 

4.6 

3.8 

3.5 

6.4 

4.2 

64 

4.0 

5.0 

5.0 

8.0 

7.0 

8.0 

7.0 

8.0 

8.0 

48 

5.0 

5.0 

6.0 

7.0 

5.0 

6.0 

6.0 

5.0 

6.0 

Feedback Quality 




52 

3.3 

7.3 

5.7 

4.7 

4.7 

4.3 

4.8 

6.6 

6.5 

64 

3.0 

6.0 

4.0 

6.0 

6.0 

10.0 

8.0 

6.0 

9.9 

48 

6.0 

5.0 

6.0 

6.0 

5.0 

6.0 

6.0 

5.0 

6.0 

Effective Input 




52 

5.1 

5.9 

5.6 

5.9 

4.6 

6.9 

5.1 

7.0 

6.1 

64 

3.0 

5.0 

5.0 

6.0 

9.0 

9.0 

7.0 

7.0 

9.0 ; 

48 

6.0 

5.0 

6.0 

6.0 

6.0 

6.0 

6.0 

5.0 

6.0 

Overall Involvment 




52 

3.2 

6.0 

5.5 

5.1 

4.5 

7.2 

5.1 

6.7 

6.3 

64 

3.0 

3.0 

3.9 

8.0 

10.0 

8.9 

8.0 

8.0 

8.9 

48 

6.0 

5.0 

6.0 

6.0 

6.0 

6.0 

6.0 

6.0 

6.0 


Overall Leader 

52 

64 

48 

52 

4.9 

5.2 

6.7 

64 

3.0 

10.0 

9.0 

48 

5.0 

6.0 

5.0 


Toolset 

Elements 

A 

Compass and Satellite Image 

Key Feature Location with Photos 
Compass Rose and Contour Map 

B 

C 
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G. PARTICIPANT PERSONALITY TRAIT SCORES 

For privacy concerns, the data received from each participant was not published 
with this document. For future research requiring the use of this data, the Principle 
Investigator, LT Krist Norlander, USNR, or the Thesis Advisor, Dr. Rudolph P. Darken, 
should be contacted at The MOVES Institute, Naval Postgraduate School, Monterey, 
California for official release of this information. 
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